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ABSTRACT 



The Division of Adult Education in the Arizona Department of 
Education and the Arizona Adult Literacy and Technology Resource Center 
contracted with the Morrison Institute for Public Policy to develop an 
evaluation system for Arizona's adult education program. During phase I, the 
institute analyzed existing program data on Arizona’s adult education 
providers. Interviews were conducted in January and February 1997 of 379 
current and past program participants; they were done in English or Spanish, 
as appropriate. Interviewees were asked 10 questions about reasons for 
entering the program, amount of time spent in the program, and feelings about 
what the program did or did not help them accomplish. Past participants were 
asked an additional question about their reasons for leaving the program. 
Findings indicated that over 90 percent of interviewees felt their adult 
education experiences had met their needs and been positive overall. The 
proportions of participants who mentioned having entered with a specific 
goal, e.g., preparing to take the General Educational Development tests or to 
get a job, were closely aligned with the proportions who said their 
participation in the program had helped them reach that particular goal . More 
than one-quarter of program participants in 1996 showed improvement on three 
indicators that suggested they were becoming more self-sufficient: moving off 
public assistance, gaining employment, and maintaining or improving their job 
situation. (Appendixes contain 10 references, 3 data tables, and interview 
forms.) ( YLB) 
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Executive Summary 



Research has affirmed that literacy skills reduce an individual’s risk of falling into dependency 
and enhance the ability to become economically self-sustaining. Adult education programs 
that develop such skills are particularly important now, as welfare reform focuses more 
attention on the most effective means of moving people toward self-sufficiency. Arizo na, 
home to thousands of individuals with insufficient language and mathematics skills to function 
effectively in society, is working to develop a system to assess the effects of its adult education 
program on its more than 50,000 annual program participants. The intent is to shift the 
current evaluation emphasis on process data to outcome data such as jobs obtained, diplomas 
earned, and participants’ satisfaction with their course work. In other words, the god of a new 
evaluation system will be to determine the impact of the adult education program. 

The Arizona Department of Education (ADE) and the Arizona Adult Literacy and 
Technology Resource Center (AALTRC) commissioned the Morrison Institute for Public 
Policy at Arizona State University to analyze existing program data on Arizona’s some 60 
adult education providers. In addition, interviews were conducted in January and February 
1997 of 379 current and past program participants. The interviews were done either in English 
or in Spanish as appropriate. Although the group of interviewees did not represent a 
scientifically random sample, the diversity of their backgrounds, needs, and program 
experiences does produce a fairly comprehensive picture of the program’s impact on 
participants. 

Interviewees were asked ten questions about their reasons for entering the adult education 
program, the amount of time spent in the program, and their feelings about what the program 
did or did not help them accomplish. Past participants were asked an additional question 
about their reasons for leaving the program. Of the 379 program participants interviewed, 
over 90% indicated that their adult education experiences had met their needs and had been 
positive overall. The proportions of participants who mentioned having entered with a 
specific goal — e.g., preparing to take the GED or to get a job — were closely aligned with the 
proportions who said that their participation in the program had helped them reach that 
particular goal. 

The new information from participant interviews generally affirms conclusions that can be 
drawn from program data published annually by ADE. Among other things, these data reveal 
that during the 1996 program year, nearly three-quarters of those who successfully completed 
the class level at which they started moved on to more advanced classes, and nearly 20% of all 
participants attained an educational “milestone” such as an adult high school diploma or 
equivalent. 

In the next phase of this project, a long-term evaluation system for Arizona’s adult education 
program will be designed and data collection instruments will be developed, based in part on 
the data collected in Phase I. Moreover, consideration will be given during Phase II to other 
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states whose experiences with measuring the impacts of adult education may prove instructive. 
Phase HI will consist of a field test of the new Arizona system and refinement of same. Given 
the current urgency to demonstrate accountability in providing education and welfare, 

Arizona must be able to document the results of its efforts to move people toward productive 
participation in society. 
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Overview 



In December 1996, the Division of Adult Education, Arizona Department of Education 
(ADE) and the Arizona Adult Literacy and Technology Resource Center (AALTRC) 
contracted with the Morrison Institute for Public Policy (MI) at Arizona State University to 
develop an evaluation system for Arizona’s Adult Education Program. Tasks include a review 
of program data for the past several years in order to identify trends in adult education 
programs, and the development and implementation of an evaluation system that will 
continue to demonstrate the aggregate impact and value of the program. 

Historically, the evaluation emphasis in Arizona’s adult education program has been on 
process data — the number of participants, number of instruction hours, and other program 
inputs. The Division of Adult Education has recently instituted activities to underscore their 
focus on the provision of quality services. The intent is to shift the evaluation emphasis to 
include outcome data such as jobs and job advancements obtained, diplomas earned through 
the General Educational Development (GED) test, and participants’ overall satisfaction with 
their adult education course work. 

The evaluation system to be developed by MI, working in collaboration with AALTRC, will 
enable the Department of Education to assess outcomes of adult education programs and to 
produce reliable data on the impact of those programs. The development of the evaluation 
system is being conducted in three phases: Phase I, Analysis; Phase II, Design and 
Development; and Phase 1H, Implementation and Evaluation. 

This report describes Phase I of the project, which included the design and analysis of 
participant interviews conducted by program providers, and an examination of existing 
process data and outcomes as described in ADE annual performance reports. It provides 
AALTRC and ADE with an analysis of available data on adult education programs statewide, 
and an assessment of those programs as compared to programs elsewhere in the United States. 
In a Preliminary Data Report (February 1997), Morrison Institute researchers provided 
AALTRC and ADE with a summary and brief discussion of data collected in interviews 
during January and February 1997 with current and past participants in projects offered by 
some 60 adult education providers overseen by ADE’s Adult Education Division. The current 
report includes the complete results of those participant interviews and discusses the 
implications of the results for Arizona’s adult education program. It should be noted, 
however, that this is not intended to be the definitive report on the adult education program. 
Rather, as specified in the agreement among AALTRC, ADE, and MI, this Phase I report is 
intended to lay the groundwork for Phase II of the project, the design and development of a 
statewide evaluation system. 
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Literacy and Welfare Reform: 
A National Perspective 



As welfare reform is implemented across the nation, considerable public attention is centered 
on how best to assist people in moving along a path toward economic self-sufficiency. With a 
focus on improving literacy and basic skills, adult education programs will be important 
contributors to the success of this process. A recent national report (Barton and Jenkins, 1995) 
explored the relationship between literacy and welfare dependence, analyzing data from the 
1992 National Adult Literacy Survey. Predictably, these two issues are closely intertwined. 
People with higher literacy competencies do better economically: they work for longer 
periods of time, at a higher average weekly wage, yielding a higher annual income than people 
with lower literacy skills. Overall, levels of success in the labor market are linked to a great 
degree with literacy levels. This relationship is true for welfare recipients as well as the general 
population. 

The presence of literacy skills moderates the degree of risk of falling into dependency as well 
as the capacity for improving self-sufficiency. One of the principal messages of the study 
underscores this connection: “Welfare dependency can be reduced in two ways: by increasing 
literacy levels in the general population... and by raising the literacy levels of those already on 
welfare to help them become more financially self-sufficient.” (Barton and Jenkins, p. 60.) 

The implications of this relationship are apparent. In Arizona, improving the literacy and 
basic education skills of the 500,000 adults without high school diplomas and the thousands of 
citizens with insufficient skills to function effectively has both individual and statewide 
impact. As Arizona invests in moving people from welfare to work and in helping low- 
income workers improve their ability to be economically self-sufficient, positive outcomes for 
participants in the state’s adult education programs will be a key factor in its success. 
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Adult Education in Arizona 



Adult education has been a concern in the state of Arizona for several decades. A testing 
service and certificate program were begun in the 1940s, and in 1965 the State Board of 
Education approved a State Plan for a formal program of adult education instruction. In 
addition to GED preparation classes, Adult Basic Education (ABE), English for Speakers of 
Other Languages (ESOL), and citizenship classes are offered. In fiscal year 1996, the various 
programs funded through the Arizona Department of Education/Division of Adult Education 
served 56,242 people, a 40% increase from fiscal year 1990. (Arizona Adult Literacy and 
Technology Resource Center, September 1996.) 

The “student body” for adult education programs in Arizona— as throughout the United 
States— is demographically diverse. More than half of the participants in the state program are 
of Hispanic origin. Of the 56,242 individuals served in FY 1996, 34,565 (61%) were Hispanic; 
12,843 (23%) were non-Hispaiuc Caucasian; 3,238 (6%) were American Indian/ Alaskan 
Native; 2,835 (5%) were Asian or Pacific Islander; and 2,761 (5%) were non-Hispanic Blacks. 
Enrollees in the Arizona program tend more often to be female and middle-aged or younger. 

In FY 1996, women outnumbered men, 31,354 (56%) to 24,888 (44%). Participants aged 16-24 
numbered 23,527 (42%) and those aged 25-44 were 26,973 (48%); participants aged 45 and 
older numbered 5,742 (10%). (See Appendix A for demographic and program trend data.) 



The Impact of Arizona’s Adult Education Program on Participants 

The Arizona Adult Education Program mission states that “programs will assure that adults 
have access to quality educational opportunities to bridge and support them in their 
employment, job training, and higher educational aspirations, and acquire the knowledge and 
skills necessary for effective participation in society.” Two methods of assessing the impact of 
adult education classes on program participants were utilized in the current analysis: (1) an 
interview questionnaire specifically developed for this project to elicit the perceptions of 
current and past participants in the Arizona adult education program; and (2) an analysis of 
statistical data gathered and reported annually by the Division of Adult Education. 

Interview Data: Participants’ Perceptions of Program Impact 

It should be noted that, given time constraints, a true random sample of Arizona adult 
education program participants was not attempted during this phase of the project. Thus, 
although some random sampling procedures were employed in developing the interview pool, 
the data and analysis presented below do not reflect a scientifically random sample. 
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Interview Procedures 



The evaluation plan called for telephone or in-person interviews of current and past 
participants in the adult education program to be conducted by a local program provider staff 
member at as many as possible of the 61 projects in Arizona that offer adult education. 
Interview forms were developed by Morrison Institute and approved by Division of Adult 
Education administrators. Ten individuals per site were randomly selected by Morrison 
Institute for interviews: five from among FY 1997 participants to gather data on the quality of 
current programs and five from among FY 1996 students who are no longer enrolled (i.e., past 
participants) to provide information on reasons that students stop attending adult education 
classes. The Director of the Division of Adult Education sent a 

letter to the program directors or coordinators at all 61 projects requesting that they conduct the 
interviews and Morrison Institute followed up with a fax asking that the projects provide lists of 
all local FY 1996 and FY 1997 program participants to the Institute so a sample could be drawn. 
Local programs with very large numbers of participants were allowed to submit 
representative samples of 200 names each from their 1997 and 1996 lists to reduce staff time 
and paperwork. One program was excluded from the study at its request because it serves 
severely mentally impaired adults. Participant lists were provided as requested from 54 of the 
remaining 60 local projects, a 90 percent response rate. 

Morrison Institute staff used the participant lists to select a sample of 20 individuals from each 

1996 and 1997 list. Each sample represented the five participants from each list to be called 
first for interviews, plus 15 potential replacement interviewees for participants who could not 
be reached after three calls to their home or workplace telephone number of record. The two 
lists (FY 1996 and FY 1997) of randomly selected interviewees and replacement interviewees 
for each project were then sent back to the providers. The interview forms and directions for 
local project personnel to conduct the interviews (see Appendix B) were sent with the 
interviewee lists. Interviews were to be conducted in either English or Spanish as needed. 

Morrison Institute mailed the interviewee lists and forms to the 54 projects between January 
23 and February 14, 1997 with the request that the completed forms be returned to MI by 
February 26. Interview data were received from a total of 46 projects. The 46 projects 
represent 77 percent of the 60 selected Adult Education Program projects in the statewide 
program and 85 percent of the 54 projects that provided Morrison Institute with FY 1996 and 

1997 participant lists. Several of the remaining eight projects informed MI staff that they were 
unable to complete their interviews by the deadline. It is not known how — if at all — interview 
data from the 14 program providers who did not return completed interviews might have 
changed overall results. Thus, it must be kept in mind that the data provided by the 46 
projects who responded may present an exaggeratedly positive picture of adult education in 
Arizona. 

Respondents 

Interview data were received on 379 present and former students — 213 from FY 1997 and 166 
from FY 1996— from the 46 projects. One hundred thirty-six interviewees were male and 243 
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were female. Of the 364 interviewees who gave their ages, 131 were under age 25, 173 were 
between the ages of 25 and 44, and 60 were 45 or above. Of the 373 interviewees who 
provided information on their ethnic backgrounds, 219 interviewees were Hispanic, 77 were 
Caucasian, 53 Native American, 16 African-American, 6 Asian/Pacific, and one each Syrian 
and “mixed.” Although interviewers were more successful at reaching women and older 
individuals, these proportions closely correspond to the demographic profile of Arizona adult 
education program participants. 

Results 

The interview results are itemized and summarized on page 6. The results are described below 
under topic headings that correspond to items or item groupings on the survey. 

Personal Goals and Guidance 

Three questions on the interview form (items 1-3) dealt with the interviewees’ personal goals 
and the guidance they received from a program representative. The questions were “Why did 
you enter the Adult Education Program?”; “Who did you talk to when you started the 
program?”; and “Did someone help you figure out what you should learn from the program?” 

Participants who had more than one reason for entering the program were told to give each 
reason. It can be seen from the summary data that “Learn to read/ write English better” was 
the most common reason, cited by 200 of the 379 participants (53%). Other frequently 
reported reasons were “Prepare for the GED” (181 participants, 48%), “Get a job or get a 
better job” (133 participants, 35%), and “Get into a school or training program” (72 
participants, 19%). 

With regard to guidance from a program representative, 353 of the interviewees (93%) 
reported that they talked to a program representative such as the director or coordinator, an 
instructor, or a staff member, at the time they started the program. Three hundred eleven of 
these participants who said they got help (98%) stated that a program representative helped 
them figure out what they should learn from the program. 

Time in Program 

Interviewees were asked two questions (items 6 and 7) about their time in the program. The 
most common responses to the question “How long [have you been/ were you] in the Adult 
Education Program?” were 1 to 3 months (111 of 379 respondents, or 29%) and 3 to 6 months 
(96 respondents, 25%). Fifty-seven participants (15%) reported that they had been in the 
program more than a year. The most common number of hours of class time logged by 
participants in the program were 10 to 50, reported by 136 participants (36%). Ninety-eight 
respondents (26%) reported having more than 100 hours of class time. 
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Adult Education Program: Summary Data from Participant Interviews 



The data below are from interviews conducted by Arizona Adult Education Program providers with 379 randomly selected program 
participants in the FY 1996 and FY 1997 programs at 46 local projects throughout the state. The numbers in italics represent the number and 
percentage of interviewees giving the designated response. Not all respondents answered all questions; percentages given are of the total 
number of respondents to each question. 



1. Why did you enter the Adult Education Program? (Multiple responses were accepted) 

200 (53%) Learn to read/write English better 133 (35%) Get a job or get a better job 

59 ( 16%) Learn math better 72 (19%) Get into a school or training program 

181 ( 48% ) Prepare for the GED 56 (15%) Prepare for citizenship 

74(20%) Other 

2. Who did you talk to when you started the program? 

353 (96%) Program representative (Director, instructor, staff member, etc.) 

15 (4%) Other (Relative, friend, employer, etc.) or no one 

3. Did someone help you figure out what you should learn from the program? 

Yes 318(84%) No 59(16%) If yes, who? Program representative 311(98%) 

Other 6 (2%) 

4. Do you think the class [you’re taking/you took] [is/was] a good one? 

Yes 363(97%) No 11(3%) 

5. [Does/Did] the class cover what you [want/wanted] to learn? 

Yes 351(93%) No 25(7%) 

6. How long [have you been/were you] in the Adult Education Program? 

1 month 64 1 to 3 HI 3 to 6 96 6 months 51 More than 57 

or less (17%) months (29%) months (25%) to 1 year (14%) lyear (15%) 

7. About how many hours of class [have you had/did you have] in the Adult Education Program? 

Less than 10 42(11%) 10 to 50 

8. [Is/Did] the program [helping/help] you? 

Yes 352(94%) No 23(6%) 

If yes, how? (multiple answers accepted) 

207 (59%) Learn English better 

93 (26%) Learn math better 

156 (44%) Prepare for the GED 

89 (25%) Other 

9. [Is/Was] the program a good one for you? 

Yes 364(96%) No 15(4%) 

10. FY ‘96 only: Why did you stop attending the program? (Multiple answers were accepted) 

63 of 166 (38%) Got what 1 needed/Accomplished my goals 7 of 166 (4%) Not appropriate for me 

41 of 166 (25%) Took a job 7 of 166 (4%) Enrolled in another school 

11 of 166 (7%) Lost interest in it 30 of 166 (18%) Transportation or time problems 

48 of 166 (29%) Other 



136(36%) 50 to 100 101(27%) More than 100 98(26%) 



57 ( 16%) Prepare for citizenship 

126 (36%) Prepare to get a job or better job 

71 (20%) Get into a school or training program 



How could we make the program better for you? Is there anything else you’d like us to know? 



144 (38% ) Happy/satisfied/grateful for program and/or instructors 
54 ( 14% ) Increase duration, frequency and/or number of classes 

24 (6%) Expand computer capacity and availability 

23 (6%) Provide more personalized instruction 

12 (3% ) More/better books and/or materials 
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Program Value 



Four interview questions (items 4, 5, 8, and 9) dealt with the value of the Adult Education 
Program to the interviewees. These questions asked whether the classes that the interviewee is 
taking (or has taken, in the case of past participants) are good, whether they cover what the 
interviewee wants (or wanted) to learn, whether the program is helping (or did help) the 
interviewee, and whether the program is (or was) a good one for the interviewee. 

Over 90 percent of the interviewees responded positively to each of the four questions related 
to the value of the program or of the classes in it. A total of 363 of the 379 interviewees (96%) 
reported that their class(es) are (or were) good ones, and 351 participants (93%) indicated that 
the class(es) cover or covered what they wanted to learn. Three hundred fifty-two students 
(93%) responded “yes” to the question about whether the program is helping (or did help) 
them, and 364 (96%) stated that the program is (or was) a good one for them. When students 
who said that the program was helping or had helped them were asked “How?,” 59 percent 
responded that it helped (or is helping) them learn English better, 44 percent said it helped 
them prepare for the GED, 36 percent indicated that it helped them get a job or get a better 
job, 26 percent that it helped them learn math better, and 20 percent that it helped them get 
into a school or training program. 

Termination of Attendance 

Participants from FY 1996 were asked why they stopped attending the program. Sixty-three of 
the 166 FY 1996 participants (38%) replied that they either had gotten what they needed or 
had accomplished their goals. Forty-one respondents (25%) reported that they stopped 
attending because they got a job and 30 (18%) left because of transportation or time problems. 
Of the 48 respondents who gave “Other” reasons for leaving, 15 (9%) specifically cited 
personal problems, health problems, or family responsibilities. 

Interviewee Comments 

Interviewees were also asked two open-ended questions: “How could we make the program 
better for you?” and “Is there anything else you’d like us to know?” Responses that were 
similar from two or more interviewees were summarized into a single response category, 
rather than being listed individually, to facilitate reporting and comprehension of the results. 
By far the most common response given by 144 of the 379 participants (38%), was that the 
interviewee was happy, satisfied, or grateful for the program and/or the instructors. Other 
responses made by more than ten participants were “Increase the duration, frequency, and/ or 
number of classes” (54 participants or 14%), “Expand the computer capacity and/or 
availability” (24 participants, 6%), “Provide more personalized instruction” (23 participants, 
6%), and “Provide more/better books and/or materials” (12 participants, 3%). 
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Discussion 



The 379 participant interviews reveal favorable perceptions of the Arizona Adult Education 
Program. The most impressive data are the results for the four items dealing with the value of 
the program and the classes in it. Participants consistently reported that the program classes 
are good, that they cover what the participants want to learn, that the classes are helping 
them, and that the program is a good one for them. 

Participant perceptions of how the program is helping them corresponded very closely with 
their reported personal goals or reasons for entering the program. This correspondence can be 
seen below in the comparison of the top four reasons given for entering the program and the 
responses later in the interview to the question “How?” asked of the 352 respondents who 
reported that the program is helping or did help them. 

Top Reasons for Entering Program How Program Is Helping 

53% Learn to read/ write English better 59% Learn English better 

48% Prepare for the GED 44% Prepare for the GED 

35% Get a job or get a better job 36% Prepare to get a job/better job 

19% Get into a school or training program 20% Get into a school or training program 

The close correspondence in percentages for each of these items seems to indicate that the 
learning opportunities provided in the program successfully address participants’ primary 
reasons for entering it. 

The two most common reasons reported by FY 1996 interviewees for terminating 
participation in the program also appear to indicate positive accomplishments for the 
participants and the program. Thirty-eight percent of the participants reported that they 
stopped attendance because they accomplished their goals or got what they needed from the 
program. Twenty-five percent stated that they stopped attending because they took a job. Less 
frequently cited reasons for terminating included transportation problems, time demands, 
personal and family matters, and loss of interest in the program. 

Interviewee responses to the two open-ended questions at the end of the interview generally 
support the favorable perceptions of the Adult Education Program reflected in the other 
interview data. Thirty-eight percent of the participants took this opportunity to express 
positive reactions to the program and/ or the instructors. However, many respondents also 
expressed a desire for expanded program services. A combined total of nearly 30 percent of the 
interviewees indicated that their program could be improved by: 

• more and longer classes 

• increased computer access and capabilities 

• more personalized instruction 

• improved or more plentiful texts and materials. 

15 
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Verification of Interview Data 



The data presented in this report were gathered by personnel at Adult Education Program 
local projects in interviews conducted with known participants from their own local projects. 
This procedure was used in order to involve the local projects, ensure the cooperation of local 
participants in the interviews, and collect a sufficient amount of data during the short duration 
of the project. However, data are often less reliable when the data collectors and interviewees 
may know each other and/ or when the data collectors may have a special interest in the 
nature of the results. Further development of an evaluation system for the Arizona Adult 
Education Program should move toward a process in which meaningful evaluation data are 
collected systematically as a part of the ongoing education program. 

To help address the issue of the reliability of the interview data, follow-up verification of the 
interview results was conducted with a sub-sample of interviewees. Using an alphabetized list 
of the 46 program providers who submitted completed interviews, evaluators selected every 
second provider for follow-up. For each of the 23 providers composing the sub-sample, the 
intent was to contact one current and one past participant by telephone to verify their original 
interview responses; that is, to check on data that were reported by adult education program 
providers. Five providers had no pool of past participants, resulting in a sub-sample of 41 
possible contacts. 

Follow-up telephone calls were made between February 21 and February 27, 1997, including 
evenings and a weekend. The calls were conducted by a bilingual member of the evaluation 
team who was able to query participants in either English or Spanish. When reached, each 
participant was told that the evaluation team wanted to ensure that her/his views on the adult 
education program were accurately and completely reflected. A script that included open- 
ended questions was used to elicit responses that would either confirm or dispute the 
responses recorded by the original interviewer. In the few instances in which reported data 
were not confirmed by participants, efforts were made to uncover possible interviewer-specific 
confusion or misreporting. 

Of the original 41 possible contacts, seven participants either had no telephone or had an 
invalid number, and two could not be reached despite numerous attempts. Of the remaining 
32 follow-up calls, 29 participants confirmed that the information recorded on their respective 
interview forms was accurate. Two participants could not remember having been interviewed 
and one participant indicated that she/he had made some negative remarks that were not 
recorded on the interview form. As a result of these apparent discrepancies, telephone calls 
were made to four additional participants who had been interviewed by the same 
interviewees) as the three participants whose interview responses had not been confirmed. In 
each of these subsequent calls, participants confirmed that the information recorded on their 
respective interview forms was accurate. 

In summary, 36 telephone calls were successfully completed to current and past participants 
across 23 adult education projects. Of these, 33 participants (92%) confirmed the accuracy of 
the information recorded on their interview forms. 
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ADE Annual Performance Data: Statistical Reflection of Program Impact' 



The impact of the adult education program on individual participants is also illustrated by 
tracking their progression through increasingly higher levels of education and training; their 
attainment of educational “milestones;” and, ultimately, the effect that this expanded 
education and training have on moving them toward economic self-sufficiency. 

Progressing to Higher Levels of Education and Training 

Students identified in Arizona’s adult education annual performance reports as having “moved 
to a higher level” of education or training are those who, for example, complete ABE classes 
and continue on with a GED class, or are ESOL students who move into GED-Prep classes. 
For those participants completing GED-Prep classes, higher level progress usually means 
enrollment in a community college, university, or job training program. 



Table 1 Adult Education Students Progressing to Higher Levels of Education/Training: 1996 





ABE (I & II) 


ESOL (I, II, III) 


GED Prep 


Number of students starting classes at this level 


16,081 


25,138 


12,962 


Percent of students completing these classes* 


4294 (27%) 


5320 (21%) 


3843 (30%) 


Percent of students completing these classes who 
went on to higher level of education/training* 


3073 (72%) 


3613 (68%) 


1727 (45%) 



Data in this table were derived from Table 3 in the amended FY 1996 Arizona Adult Education Student Data Report. 
•Rounded to nearest whole percent 



Many students in the state’s adult education program appear to be making progress along this 
education/training continuum. During the 1996 program year, more than 25 percent of 
students in ABE classes successfully completed the class level at which they started; of these 
students, nearly three-quarters (72%) proceeded on to more advanced-level classes. For those 
students beginning the year in ESOL classes, 21 percent completed the level at which they 
entered, with more than two-thirds (68%) then moving into a higher level of education. 
Similarly, 30 percent of students who started the year in GED-Prep completed the class, and 
nearly half of these students (45%) continued their education or training. 



*Data used to develop this section come from the FY 1993-FY 1996 (updated) Arizona Adult Education Performance Reports prepared 
by the Arizona Department of Education, Division of Adult Education. 
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Reaching Educational “Milestones” 



Among the list of “student goals achieved” through participation in adult education classes, 
four in particular suggest students’ attainment of educational milestones. These are: (1) 
obtaining a high school diploma; (2) passing the GED or being in process to take the GED; (3) 
entering another educational or training program; and (4) applying for college entrance. Data 
for FY 1996 indicate that 11,258 adult education students in Arizona (20%) reported reaching 
one or more of these four milestones. Although information on “students applying for college 
entrance” was not reported for the years prior to 1996, trend data for the three remaining 
indicators are available. Between 1993 and 1996, there was a steady increase in the percentage 
of adult education participants reaching one or more of these educational milestones, rising 
from 11.8 percent to 18.3 percent during this time period. 



Table 2 Adult Education Students Achieving Educational “Milestones”: 1993-1996 




1993 


1994 


1995 


1996 


: Total # m education students' .||§|j Jf : ff|jj '! 


■ 44,m 


ililllll 




i 56,242. | 


Obtained adult high school diploma 


54 


89 


6l 


60 


Passed GED/in progress 


4347 


4776 


6210 


7320 


Entered other education/job program 


879 


2828 


2459 


2930 


Applied for college entry 


N/A 


N/A 


N/A 


[948]* 


Total percent of students achieving educational 
milestones 


11.8% 


15.9% 


16.6% 


18.3% 



Data in this table were derived from Table 6 in the FY 1993-FY 1996 (updated) Arizona Adult Education Annual Performance Reports 
*Not included in calculation of total percentage 
N/A=not available 



Reducing Welfare Dependency and Improving Employment 

One way to look at the progress made by adult education program participants is by 
reviewing “student goals achieved” during the program year as they relate to participant status 
recorded upon program entry. Three sets of items from these categories specifically relate to 
participants’ progress toward self-sufficiency: (1) adults on public assistance/ removed from 
public assistance; (2) unemployed adults/ gained employment; and (3) employed adults/ secured 
employment retention or obtained job advancement (see Table 3, page 12). 
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Table 3 Adult Education Participant Status: Reduction in Welfare Dependency and/or 
Improvement in Employment: 1994-1996 




1994 


1995 


1996 


Participants on public assistance upon entry who went off 
public assistance during the year 


4% 


7% 


8% 


Participants unemployed at entry who gained employment 
during the year 


9 % 


7% 


6% 


Participants employed on entry who maintained or 
improved their job situation 


9 % 


11% 


14% 


Total* 


22 % 


25% 


28% 



* Since participant gains on these indicators are closely linked, data presented in this table likely reflect duplicated counts. 

Data were calculated using Tables 5 and 6 in the FY 1994-FY 1996 (updated) Arizona Adult Education Annual Performance Reports. 



More than one-quarter of adult education program participants in 1996 showed improvement 
on the three indicators that would suggest they were becoming more self-sufficient: students 
moving off public assistance, students gaining employment, and— for those students already 
employed upon entry— maintaining or improving their job situation. Overall for these three 
indicators, there has been small but steady progress between 1994 and 1996, from a total of 22 
percent of program participants showing improvement in 1994 to a total of 28 percent in 
1996. It should be noted that since participant gains on these indicators are often linked, the 
overall progress likely reflects some degree of overlap. Although the percentage of 
unemployed students who got jobs decreased slightly during this period, increases were 
recorded in the percentage of participants who left the welfare rolls and those who maintained 
or improved their employment. 
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Arizona in a National Context 



The overall picture of adult education in the United States is unclear because of information 
that varies widely in completeness and accuracy from state to state. In 1991, the most recent 
year for which published figures are available, the U.S. Department of Education (USDOE) 
could estimate the total number of individuals served by programs funded under the Adult 
Education Act (AEA; re-authorized in 1990) no more precisely than within the range of 2.6 
million and 3.7 million people (U.S. Department of Education, 1995). 

Among the findings of the 1993-94 national report was that enrollment in ESOL classes 
nationwide had risen dramatically in recent years as a proportion of enrollment in all adult 
education programs. Available data on Arizona’s experience, however, show that the 
proportion remained roughly the same in 1996 as in 1990. 

In attempting to assess the quality of adult education programs across the nation, the USDOE 
takes the following factors into account, as enumerated in the 1993-1994 biennial report: 

• recruitment 

• retention 

• educational gains 

• program planning 

• curriculum and instruction 

• staff development 

• support services 

The USDOE has explicitly noted the wide variations among state-level data collection and 
analysis procedures and the consequent failure of a USDOE-developed evaluation model to be 
applicable to all states. Nonetheless, currently available quantitative data and qualitative 
information from Arizona’s adult education providers can be assessed in relationship to some 
of those national-level factors. 

• Recruitment. Generally, neither the Arizona State Department of Education nor individual 
adult education program providers engage actively in recruitment efforts. The more than 
doubling of the number of participants within the past decade (25,330 participants in FY 
1987; 56,242 in FY 1996) has been attributed by ADE administrators in part to “word-of- 
mouth” advertising. 

• Retention. The record of Arizona’s adult education program at retaining participants once 
they have enrolled has been addressed previously, in the analysis of students’ progress 
across educational levels and their achievement of educational “milestones.” In its FY 1993- 
1994 report, the USDOE noted that 33% of adult education program participants nationwide 
remained in those programs long enough to progress one instructional level; in that same 
period, 26% of Arizona program participants completed one full instructional level. 
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Concern has been voiced on the national level that inadequate or nonexistent support 
services such as transportation and child care have a seriously negative impact on 
participant retention. Data indicate that such factors can also be significant for Arizona 
participants when they are deciding whether or not to continue in state adult education 
programs. Among the reasons given by interviewed past participants for leaving Arizona 
adult education programs, the two most frequently cited were “Got what I needed” (35%) 
and “Took a job” (25%), with “Transportation or time problems” coming in third (18%). 

In ADE annual report data, the three reasons cited most often — by far — by participants for 
leaving adult education programs are “Took a job,” “changed address/ moved,” and “lack of 
interest.” AALTRC administrators indicated that program providers tend to use “lack of 
interest” as the “default” category when they do not specifically know the reasons for 
participants’ departures. 

• Educational Gains. As noted above, interviews of current and past participants in the 
Arizona adult education program focused heavily on the gains that individual participants 
had, in their own views, made as a result of their participation. Of those who responded 
that their programs had helped them, nearly half (44%) said that they had been adequately 
prepared to take the GED; 20 percent said that participation in the program had helped get 
them into a school or a training program; more than half (59%) said they had improved 
their knowledge of the English language; and 26 percent claimed that their mathematical 
abilities had improved. 

• Program planning curriculum and instruction, and staff development vary from provider to 
provider and from project to project within the state, and there currently are no consistent 
statewide standards. Arizona, through the Department of Education, has implemeted a 
staff development initiative by providing funding, earmarked for this purpose, directly to 
local projects. ADE also provides special curriculum workshops and funds promising 
practices with discretionary funds. In addition, ADE sponsors statewide adult education 
conferences. With the recent development and implementation of a self-assessment process, 
projects are starting to focus on program outcomes. 

Consideration also was given to a number of individual states that might facilitate a 
comparative assessment of Arizona’s adult education efforts. Five states were awarded federal 
grants in October 1993 to develop outcome measurement and reporting systems for their 
adult education programs. Of them, Virginia and Hawaii did not appear to have made great 
enough progress to allow measurement against Arizona. New York is a complicated case 
because that state does not handle data on adult education and on employment separately. 
Only Kentucky and Tennessee suggested themselves as potential exemplars. The experiences 
of these two states might prove informative during the design and development of Arizona’s 
evaluation system (Phase II of this project). 
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Phase I: Concluding Thoughts 



In addition to achieving the initial objective of laying the analytic groundwork for the second 
and third phases of the project, activities in Phase I produced some valuable insights. It must 
be noted again, however, that the results of the interviews conducted for Phase I do not 
represent a true random sample that would permit broad conclusions about Arizona’s adult 
education program. 

Suggestions from Program Providers 

In the course of the first “field-test” of the data collection procedures and interview forms 
developed for Phase I of the evaluation, MI researchers had an opportunity to speak with 
several program providers across the state. The more than 50 telephone conversations that 
took place with program providers between January 13 and February 5, 1997 offer some 
initial “logistical lessons learned” that will be taken into consideration in the design and 
development of the statewide evaluation system in Phase II of the project. 

First, the definitions of “past” and “current” program participants need to be more clearly 
delineated. A designation based on fiscal-year enrollment does not account for students 
enrolled in the current year who have left the program by the time the interview is conducted. 
The definition of past participants contained in the Division of Adult Education request-for- 
proposal — participants who are no longer attending (irrespective of the fiscal year) — would 
appear to be the simplest way to address this issue. 

Second, attention must be given to the problems described by many providers related to 
difficulties in accessing the previous year’s database. Some providers replace the previous year’s 
data in the computer, while others described difficulties due to changes in their computer 
programs. 

A final issue raised by providers was the difficulty encountered in trying to obtain data 
representative of all their project sites. This was particularly true for rural project sites that 
were not electronically “connected” to the official program provider— especially given the 
short turn-around time available for collecting the interview data in Phase I. 

Toward the Development of an Evaluation System 

An appropriate data collection and evaluation system that becomes a routine part of each 
provider’s ongoing program operations would help address issues raised during Phase I. Such 
an evaluation system would facilitate the collection of reliable data and help ensure that all 
program providers are able to submit requested data in a timely fashion. 

In the current national atmosphere surrounding reforming both welfare and education, the 
need for accountability beyond traditional fiscal and program management measures is 
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particularly urgent. If Arizona wants to succeed in its efforts to move undereducated adults 
toward economic self-sufficiency and productive participation in society, it must be able to 
document that success in reliable, accessible, and meaningful ways, not only for legislators and 
funders but, through them, for taxpayers. The time is propitious for the state to develop an 
ongoing evaluation system that can capture all relevant data about adult education program 
participants, from enrollment through graduation or separation from the program, to 
document both immediate and long-term benefits. 

Phase II of this project will focus on the design and development of such an evaluation system, 
building on the data and analysis produced in Phase I. A report on the results of Phase II will 
be completed by June 30, 1997. 
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Table A-l* Proportional Distribution of Participants by Race and 
Ethnicity: 1992-1996 





1992 


1993 


1994 


1995 


1996 


White, not Hispanic 


25% 


25% 


24% 


23% 


23% 


Black, not Hispanic 


8% 


5% 


6% 


5% 


5% 


Hispanic 


53% 


56% 


57% 


61% 


61% 


American Indian/ Alaska Native 


5% 


7% 


7% 


6% 


6% 


Asian/Pacific Islander 


9% 


7% 


6% 


5% 


5% 



Table A-2 Proportional Distribution of 
Participants by Age: 1992-1996 





1992 


1993 


1994 


1995 


1996 


16-24 years 


47% 


44% 


43% 


41% 


42% 


25-44 years 


41% 


46% 


47% 


42% 


48% 


45+ years 


12% 


10% 


10% 


17% 


10% 



Table A-3 Proportional Distribution of Participants by 
Class Type: 1992-1996 





1992 


1993 


1994 


1995 


1996 


ABE I and ABE II 


30% 


31% 


32% 


30% 


29% 


ESOL I, II, and HI 


46% 


42% 


41% 


43% 


45% 


GED Prep 


23% 


26% 


25% 


24% 


23% 


Citizenship 


1% 


1% 


2% 


3% 


3% 



♦ . 

Data in Tables A-l, A-2, and A-3 were derived from original data tables in the FY 1992-FY 1996 Arizona Adult Education Annual 
Performance Reports. 
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Appendix B 
Interview Forms 



GUIDANCE FOR COMPLETING INTERVIEWS 
WITH ADULT EDUCATION PROGRAM PARTICIPANTS 



• Ten (10) participants in your site’s program should be interviewed. Five (5) must be 
currently enrolled and five (5) must be past participants (individuals from FY ‘96) who 
either completed the program or left for other reasons. Included in this packet are lists of 
randomly-selected current participants and past participants from your site. You will be 
interviewing participants whose names appear on these two lists. 

• Begin by trying to contact the first five names on each list. Please make three attempts to 
contact each person. If you are unable to reach an individual after three attempts, proceed 
down the list in numerical order until you have successfully completed five interviews. 

• Each provider should be prepared to interview participants in their primary language as 
necessary. 

• Interviews may be conducted over the telephone or in person. Please indicate at the top of 
the interview form whether the interview took place on the telephone or in person. 

• Following submission of all interview forms, researchers will spot-check to ensure 
accuracy and completeness. 

• All ten interviews must be completed and the interview forms submitted to Morrison 
Institute by Wednesday, February 26, 1997. 
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ADULT EDUCATION PROGRAM 



INTERVIEW FORM 
FOR CURRENT PARTICIPANTS ONLY 

Note: This form is for current participants only. It is to be completed by a representative of the program 
provider based on an oral interview with a designated current participant. 

Name/Location of program provider: Phone: 

Name of interviewer 

In what language was the interview conducted? English Spanish Other (specify) 

Person interviewed: Phone: Age: 

Gender M F Ethnic Group: Afro-American Asian/Pacific 

Hispanic Native American Caucasian 



I. Why did you enter the (name of program) ? 

(Probe, if necessary, to have the interviewee state his/her reason or reasons , then check each reason 
that he/she gives below) 

Learn to read/write English better 

Leam math better 

Prepare for the GED 

Other (Please write in below) 



2. Who did you talk to at (name of program^ 

when you started the program? 



3. Did someone help you figure out what you should learn from the program? 

Yes No 

If yes, who? 

4. Do you think the class you’re taking is a good one? 

Yes No 

5. Does the class cover what you want to learn? 

Yes No 



Get a job or get a better job 

Get into a school or training program 

Prepare for citizenship 
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How long have you been in the Adult Education Program? 



1 month 1 to 3 3 to 6 6 months More than 

or less months months to a year a year 

7. About how many hours of class have you had altogether in the Adult Education Program? (Help 
interviewee estimate the total number of hours if he/she is having trouble. For example , consider weeks in 
program and hours of class per week.) 

Less than 10 10 to 50 50 to 100 More than 100 

8. Is the program helping you? 

Yes No 

If yes: How? (Have them answer and you check answers below . Check as many as they give , but don 7 
prompt them by giving possible answers.) 



Helping me leant English better 

Helping me learn math better 

Helping me prepare for the GED 

Other (please write in) 



Helping me prepare for citizenship 

Helping me prepare to get a job or get a 

better job 

Helping me get into a school or training 

program 



If no: Why not? 

Not appropriate for me Didn’t attend long enough 

Other (please write in) 



9. Is the program a good one for you? 

Yes No 

10. How could we make the program better for you? 



11. Is there anything else you’d like us to know? 
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ADULT EDUCATION PROGRAM 



INTERVIEW FORM 
FOR PAST PARTICIPANTS ONLY 

Note: This form is for past participants only. It is to be completed by a representative of the program 
provider based on an oral interview with a designated past participant. 

Name/Location of program provider: Phone: 



Name of Interviewer 

In what language was the interview conducted? English Spanish Other (Specify) 



Person interviewed: Phone: Age: 

Gender: M F Ethnic Group: Afro-American Asian/PaciFic 

Hispanic Native American Caucasian 

********************************************************************************* 

1. Why did you enter the (name of program) ? 

(Probe, if necessary , to have the interviewee state his/her reason or reasons, then check each reason that he/she 
gives below) 

Learn to read/write English better Get a job or get a better job 

Learn math better Get into a school or training program 

Prepare for the GED Prepare for citizenship 

Other (Please write in below) 



2. Who did you talk to at (name of p ro gram) 
when you started the program? 



3. Did someone help you figure out what you should learn from the program? 
Yes No 

If yes, who? 



4. Do you think the class (es) you took was/were good? 

Yes No 

5. Did the class (es) cover what you wanted to learn? 

Yes No 
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How long were you in the Adult Education Program? Was it . . . 



1 month 1 to 3 3 to 6 6 months More than 

or less months months to a year a year 

7. About how many hours of class did you have altogether in the Adult Education Program? (Help 
interviewee estimate the total number of hours if he/she is having trouble. For example , consider total weeks in the 
program and hours of class per week) 

Less than 10 10 to 50 50 to 100 More than 100 

8. Did the program help you? 

Yes No 

If yes: How? (Have them answer and you check answers below. Check as many as they give, but don 7 prompt 
them by giving possible answers.) 

Helped me learn English better 

Helped me learn math better 

Helped me prepare for the GED 

Other (please write in) 



If no: Why not? 

Not appropriate for me Didn’t attend long enough 

Other (please write in) 



9. Was the program a good one for you? 

Yes No 

10. Why did you stop attending the program? (Have them answer, then check reasons below. Check as many as 
they give, but don 7 prompt them by giving possible reasons.) 

.Not appropriate for me 
. Enrolled in another school 
.Transportation or time problems 



Got what I needed from it 
Took a job 
Lost interest in it 
Other (please write in) 



Helped me prepare for citizenship 
Helped me get a job or get a better job 
Helped me get into a school or training 
programs 



11. How could we have made the program better for you? 



12. Is there anything else you’d like us to know? 
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Morrison Institute for Public Policy analyzes current and proposed public policies 
that are important to the future of greater Phoenix, the state of Arizona, and the 
nation. Its mission is to conduct research which informs, advises, and assists 
Arizona’s state and community leaders. A unit in the School of Public Affairs (College 
of Public Programs) at Arizona State University, the Institute is a bridge between the 
university and the community. 

Morrison Institute’s services include policy research and analysis, program 
evaluation, and support of community participation in public affairs. Through a 
variety of publications and forums, the Institute shares research results and provides 
services to public officials, private sector leaders, and community members who 
shape public policy. 

Morrison Institute was established in 1981 through a grant from Marvin and June 
Morrison of Gilbert, Arizona in response to the state’s growing need for objective 
research on issues of public policy. Since then, Morrison Institute has conducted 
important work on a wide range of topics, including education reform, water 
resources, health care, human services, urban growth, government structure, arts 
and culture, technology, quality of life, public finance, the environment, and 
economic development. 

Applied public policy research that is timely, objective, and useable is Morrison 
Institute's hallmark. Consistent with this focus, the Institute annually prepares a 
practical analysis of the most important policy choices facing Arizona and its 
localities. 

Morrison Institute for Public Policy is supported by private and public funds and 
contract research. Under the auspices of Arizona State University, the Institute 
employs a staff of highly experienced researchers and routinely includes faculty 
members on project research teams. Morrison Institute is assisted in these efforts by 
a non-partisan advisory board of leading Arizona business people, scholars, public 
officials, and public policy experts. 

Morrison Institute for Public Policy 
School of Public Affairs 
Arizona State University 
Tempe, Arizona 85287-4405 
Phone: (602)965-4525 
Fax: (602)965-9219 
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